Systems and methods for determining an optimal strategy

ABSTRACT

The present disclosure is related to systems and methods for determining an optimal strategy. The method includes classify one or more users into a first user group and a second user group using an optimization model, wherein the first user group and the second user group correspond to two strategies, respectively. The method also includes obtain behavior data from terminals of the one or more users in the first user group and the second user group. The method further includes determine a first value of a parameter regarding the first user group and a second value of the parameter regarding the second user group using the optimization model. The method further includes determine a strategy based on the first value and the second value.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of International Application No.PCT/CN2018/096509, filed on Jul. 20, 2018, which further claims priorityto Chinese Patent Application No. 201710613409.4, filed on Jul. 25,2017, and Chinese Patent Application No. 201710618381.3, filed on Jul.26, 2017. Each of the above-referenced applications is expresslyincorporated herein by reference in their entireties.

TECHNICAL FIELD

This disclosure generally relates to computer based data managementsystem, and more particularly, relates to systems and methods fordetermining an optimal strategy in a strategy evaluation system.

BACKGROUND

Strategy evaluation systems are widely used in various fields, such asonline to offline services (e.g., a taxi service, a delivery service, anonline shopping service), product research and development (R&D),advertisement, etc. Among them, A/B testing, is frequently used in thestrategy evaluation systems, to evaluate performances of two strategiesbased on users' behavior data (e.g., user acceptances), and determine anoptimal strategy. This patent application provides systems and methodsto modify an NB test model in order to effectively improve the accuracyof the business strategy assessment. In addition, the A/B testing usedherein, instead of one dimension, may be in multiple dimensions (e.g.,age, gender, or education background), and thereby, providing morecomprehensive users' feedback analysis and a more reliable optimalstrategy.

SUMMARY

According to an aspect of the present disclosure, a system fordetermining an optimal strategy is provided. The system may include atleast one storage medium storing a set of instructions and at least oneprocessor configured to communicate with the at least one storagemedium. When executing the set of instructions, the at least oneprocessor is directed to perform one or more of the followingoperations, for example, classify one or more users into a first usergroup and a second user group using an optimization model, wherein thefirst user group and the second user group correspond to two strategies,respectively; obtain behavior data from terminals of the one or moreusers in the first user group and the second user group; determine,based on the behavior data of the one or more users, a first value of aparameter regarding the first user group and a second value of theparameter regarding the second user group using the optimization model;and determine an optimal strategy based on the first value and thesecond value.

In some embodiments, the at least one processor may be further directedto initiate the optimization model to classify the one or more usersinto the first user group and the second user group when the systemobtains service requests from the one or more users.

In some embodiments, to classify the one or more users into the firstuser group and the second user group, the at least one processor may bedirected to generate a random number for each of the one or more users;determine whether the random number for each of the one or more users isgreater than a threshold; in response to the random number for a userbeing greater than the threshold, classify the user into the first usergroup.

In some embodiments, in response to the random number for a user beingnot greater than the threshold, the at least one processor may befurther directed to classify the user into the second user group.

In some embodiments, the parameter may include a parameter in multipledimensions.

In some embodiments, the parameter in multiple dimensions may relate toorder information of on-demand services.

In some embodiments, the multiple dimensions may include a gender, acity, and/or an operation system of the terminal of the one or moreusers.

In some embodiments, to obtain behavior data from the terminals of theone or more users in the first user group and the second user group, theat least one processor may be directed to obtain user logs including thebehavior data from terminals of the one or more users using a HadoopDistribute File System.

In some embodiments, the at least one processor may be further directedto determine a parameter difference of the parameter regarding the firstuser group and the second user group; determine a reliability level ofthe parameter difference; and adjust the optimization model based on theparameter difference and the reliability level of the parameterdifference.

In some embodiments, to determine the parameter difference of theparameter regarding the first user group and the second user group, theat least one processor may be directed to determine a first differencevalue of the parameter regarding the first user group and the seconduser group.

In some embodiments, to determine the first difference value of theparameter regarding the first user group and the second user group, theat least one processor may be directed to determine a second differencevalue; determine a third difference value; and determine the firstdifference value based on the second difference value and the thirddifference value.

In some embodiments, to determine the second difference value, the atleast one processor may be directed to obtain behavior data of the oneor more users associated with one of the two strategies; determine,based on the behavior data of the one or more users associated with theone of the two strategies, a third value of the parameter regarding thefirst user group and a fourth value of the parameter regarding thesecond user group using the a correction model; and determine the seconddifference value based on the third value and the fourth value.

In some embodiments, to determine the third difference value, the atleast one processor may be directed to determine the third differencevalue based on the first value and the second value.

In some embodiments, to determine the reliability level of the parameterdifference, the at least one processor may be directed to determine, ata preset confidence coefficient, a confidence interval of the firstdifference value.

In some embodiments, to determine the reliability level of the parameterdifference, the at least one processor may be directed to determine a Pvalue based on the first difference value; compare the P value with asignificance value; and determine the reliability level of the parameterdifference based on the comparison of the P value with the significancevalue.

According to an aspect of the present disclosure, a system fordetermining an optimal strategy is provided. The system may include atleast one storage medium storing a set of instructions and at least oneprocessor configured to communicate with the at least one storagemedium. When executing the set of instructions, the at least oneprocessor is directed to perform one or more of the followingoperations, for example, classify one or more users into a first usergroup and a second user group using an optimization model, wherein thefirst user group and the second user group correspond to two strategies,respectively; obtain behavior data from terminals of the one or moreusers in the first user group and the second user group; determine,based on the behavior data of the one or more users, a first value of aparameter in multiple dimension regarding the first user group and asecond value of the parameter in multiple dimension regarding the seconduser group using the optimization model; and determine an optimalstrategy based on the first value and the second value.

According to another aspect of the present disclosure, a method fordetermining an optimal strategy may be determined. The method may beimplemented on a computing device having at least one processor and atleast one computer-readable storage medium. The method may include, forexample, classifying one or more users into a first user group and asecond user group using an optimization model, wherein the first usergroup and the second user group correspond to two strategies,respectively; obtaining behavior data from terminals of the one or moreusers in the first user group and the second user group; determining,based on the behavior data of the one or more users, a first value of aparameter regarding the first user group and a second value of theparameter regarding the second user group using the optimization model;and determining an optimal strategy based on the first value and thesecond value.

According to another aspect of the present disclosure, a method fordetermining an optimal strategy may be determined. The method may beimplemented on a computing device having at least one processor and atleast one computer-readable storage medium. The method may include, forexample, obtaining behavior data from terminals of the one or more usersin the first user group and the second user group; determining, based onthe behavior data of the one or more users, a first value of a parameterin multiple dimension regarding the first user group and a second valueof the parameter in multiple dimension regarding the second user groupusing the optimization model; and determining an optimal strategy basedon the first value and the second value.

According to still another aspect of the present disclosure, anon-transitory computer readable medium is provided. The non-transitorycomputer readable medium may include at least one set of instructionsfor determining an optimal strategy, wherein when executed by at leastone processor of a computer device, the at least one set of instructionscauses the computing device to perform a method. The method may include,for example, obtaining behavior data from terminals of the one or moreusers in the first user group and the second user group; determining,based on the behavior data of the one or more users, a first value of aparameter in multiple dimension regarding the first user group and asecond value of the parameter in multiple dimension regarding the seconduser group using the optimization model; and determining an optimalstrategy based on the first value and the second value.

Additional features will be set forth in part in the description whichfollows, and in part will become apparent to those skilled in the artupon examination of the following and the accompanying drawings or maybe learned by production or operation of the examples. The features ofthe present disclosure may be realized and attained by practice or useof various aspects of the methodologies, instrumentalities andcombinations set forth in the detailed examples discussed below.

BRIEF DESCRIPTION OF THE DRAWINGS

The present disclosure is further described in terms of exemplaryembodiments. These exemplary embodiments are described in detail withreference to the drawings. The drawings are not to scale. Theseembodiments are non-limiting exemplary embodiments, in which likereference numerals represent similar structures throughout the severalviews of the drawings, and wherein:

FIG. 1 is a schematic diagram illustrating an exemplary strategyevaluation system according to some embodiments of the presentdisclosure;

FIG. 2 is a schematic diagram illustrating exemplary components of acomputing device according to some embodiments of the presentdisclosure;

FIG. 3 is a schematic diagram illustrating exemplary hardware and/orsoftware components of an exemplary user terminal according to someembodiments of the present disclosure;

FIG. 4 is a schematic diagram illustrating an exemplary processingengine according to some embodiments of the present disclosure;

FIG. 5 is a schematic diagram illustrating an exemplary processingengine according to some embodiments of the present disclosure;

FIG. 6 is a flow chart illustrating an exemplary process for determiningan optimal strategy according to some embodiments of the presentdisclosure;

FIG. 7A is a schematic diagram illustrating an exemplary process fordetermining an optimal strategy according to some embodiments of thepresent disclosure;

FIG. 7B is a schematic diagram illustrating an exemplary parameter inmultiple dimensions according to some embodiments of the presentdisclosure;

FIG. 8 is a flow chart illustrating an exemplary process for classifyingone or more users into a user group according to some embodiments of thepresent disclosure;

FIG. 9 is a flow chart illustrating an exemplary process for adjustingan optimization model according to some embodiments of the presentdisclosure;

FIG. 10 is a flow chart illustrating an exemplary process fordetermining a parameter difference according to some embodiments of thepresent disclosure;

FIG. 11 is a flow chart illustrating an exemplary process fordetermining a second difference value according to some embodiments ofthe present disclosure;

FIG. 12 is a flow chart illustrating an exemplary process fordetermining a reliability level of the parameter difference based in aconfidence interval according to some embodiments of the presentdisclosure; and

FIG. 13 is a flow chart illustrating an exemplary process fordetermining a reliability level of the parameter difference based on aprobability value according to some embodiments of the presentdisclosure.

DETAILED DESCRIPTION

In order to illustrate the technical solutions related to theembodiments of the present disclosure, brief introduction of thedrawings referred to in the description of the embodiments is providedbelow. Obviously, drawings described below are only some examples orembodiments of the present disclosure. Those having ordinary skills inthe art, without further creative efforts, may apply the presentdisclosure to other similar scenarios according to these drawings.Unless stated otherwise or obvious from the context, the same referencenumeral in the drawings refers to the same structure and operation.

As used in the disclosure and the appended claims, the singular forms“a,” “an,” and “the” include plural referents unless the content clearlydictates otherwise. It will be further understood that the terms“comprises,” “comprising,” “includes,” and/or “including” when used inthe disclosure, specify the presence of stated steps and elements, butdo not preclude the presence or addition of one or more other steps andelements.

Some modules of the system may be referred to in various ways accordingto some embodiments of the present disclosure, however, any number ofdifferent modules may be used and operated in a client terminal and/or aserver. These modules are intended to be illustrative, not intended tolimit the scope of the present disclosure. Different modules may be usedin different aspects of the system and method.

According to some embodiments of the present disclosure, flow charts areused to illustrate the operations performed by the system. It is to beexpressly understood, the operations above or below may or may not beimplemented in order. Conversely, the operations may be performed ininverted order, or simultaneously. Besides, one or more other operationsmay be added to the flowcharts, or one or more operations may be omittedfrom the flowchart.

Technical solutions of the embodiments of the present disclosure bedescribed with reference to the drawings as described below. It isobvious that the described embodiments are not exhaustive and are notlimiting. Other embodiments obtained, based on the embodiments set forthin the present disclosure, by those with ordinary skill in the artwithout any creative works are within the scope of the presentdisclosure.

An aspect of the present disclosure is directed to systems and methodsfor determining an optimal strategy. The systems and methods mayclassify one or more users into a first user group and a second usergroup using an optimization model. The first user group and the seconduser group may correspond to two strategies, respectively. The systemsand methods may also obtain behavior data from terminals of the one ormore users associated with the two strategies. The systems and methodsmay further determine, based on the behavior data of the one or moreusers, a first value of a parameter regarding the first user group and asecond value of the parameter regarding the second user group using theoptimization model. Accordingly, the systems and methods may select auser group from the first user group and the second user group based onthe first value and the second value and determine a strategycorresponding to the selected user group as the optimal strategy.

Another aspect of the present disclosure is directed to systems andmethods for adjusting the optimization model. The systems and methodsmay determine a parameter difference of the parameter regarding thefirst user group and the second user group. The systems and methods mayfurther determine a reliability level of the parameter difference.Accordingly, the systems and methods may adjust the optimization modelbased on the parameter difference and the reliability level of theparameter difference.

Still another aspect of the present disclosure is directed to systemsand methods for applying a multidimensional grouping method on users anda big data analysis method on user feedback to improve the accuracy andefficiency of a strategy evaluation process. Instead of grouping theusers in one dimension, the systems and methods may group the users inmultiple dimensions (e.g., age, gender, education background, etc.).

FIG. 1 is a schematic diagram of an exemplary strategy evaluation systemaccording to some embodiments of the present disclosure. The strategyevaluation system 100 may include a server 110, a network 120, a storagedevice 130, and a user terminal 140. The strategy evaluation system 100may evaluate two or more strategies using an optimization model (e.g.,an A/B testing model). Merely for illustration purposes, the strategyevaluation system 100 may obtain certain behaviors of one or more usersassociated with a service (e.g., a taxi-hailing service, a deliveryservice, an advertisement service, an online shopping service, etc.)from the user terminal 140. The server 110 may classify the one or moreusers into two or more user groups, and the two or more user groups maycorrespond to the two or more strategies, respectively. For example, afirst user group may correspond to a strategy for adjusting the price ofthe service dynamically, and a second user group may correspond to astrategy for the price of the service remaining unchanged. The server110 may evaluate performances of the two strategies based on theobtained behaviors of the one or more users. In some embodiments, theone or more users are classified into groups according to multiplefactors, such as job, age, education background, income, etc. In someembodiments, the strategy evaluation system 100 may adjust theoptimization model based on a reliability level associated with thebehaviors of the one or more users.

The server 110 may facilitate data processing for the strategyevaluation system 100. In some embodiments, the server 110 may be asingle server or a server group. The server group may be centralized, ordistributed (e.g., server 110 may be a distributed system). In someembodiments, the server 110 may be local or remote. For example, theserver 110 may access information and/or data stored in the userterminal 140, and/or the storage device 130 via the network 120. Asanother example, the server 110 may be directly connected to the userterminal 140, and/or the storage device 130 to access stored informationand/or data. In some embodiments, the server 110 may be implemented on acloud platform. Merely by way of example, the cloud platform may includea private cloud, a public cloud, a hybrid cloud, a community cloud, adistributed cloud, an inter-cloud, a multi-cloud, or the like, or anycombination thereof. In some embodiments, the server 110 may beimplemented on a computing device 200 having one or more componentsillustrated in FIG. 2 in the present disclosure.

In some embodiments, the server 110 may include a processing engine 112.The processing engine 112 may process information and/or data to performone or more functions described in the present disclosure. For example,the processing engine 112 may obtain a service request from the userterminal 140 of one or more users, and classify the one or more usersinto a first user group and a second user group according to a usergrouping method. The first user group and the second user group maycorresponding to two strategies, respectively As another example, theprocessing engine 112 may obtain behavior data of the one or more users(e.g., operation logs of the user terminal 140), determine, in one ormore dimensions based on behavior data of the one or more users, a firstvalue of a parameter (e.g., the number of service orders finished duringa certain time period) regarding the first user group and a second valueof the parameter regarding the second user group. The first value andthe second value may be used to evaluate the two strategies to determinean optimal strategy. As still another example, the processing engine 112may determine a parameter difference of the parameter regarding thefirst user group and the second user group based on the first value andthe second value. As still another example, the processing engine 112may determine a reliability level of the parameter difference. Theparameter difference of the parameter and the reliability of theparameter difference may be used to evaluate the optimization model. Insome embodiments, the processing engine 112 may include one or moreprocessing engines (e.g., single-core processing engine(s) or multi-coreprocessor(s)). Merely by way of example, the processing engine 112 mayinclude one or more hardware processors, such as a central processingunit (CPU), an application-specific integrated circuit (ASIC), anapplication-specific instruction-set processor (ASIP), a graphicsprocessing unit (GPU), a physics processing unit (PPU), a digital signalprocessor (DSP), a field-programmable gate array (FPGA), a programmablelogic device (PLD), a controller, a microcontroller unit, a reducedinstruction-set computer (RISC), a microprocessor, or the like, or anycombination thereof.

The network 120 may facilitate the exchange of information and/or data.In some embodiments, one or more components in the strategy evaluationsystem 100 (e.g., the server 110, the storage device 130, and the userterminal 140) may send information and/or data to other component(s) inthe strategy evaluation system 100 via the network 120. For example, theprocessing engine 112 may obtain behavior data of one or more users fromthe storage device 130 and/or the user terminal 140 via the network 120.In some embodiments, the network 120 may be any type of wired orwireless network, or a combination thereof. Merely by way of example,the network 120 may include a cable network, a wireline network, anoptical fiber network, a telecommunications network, an intranet, theInternet, a local area network (LAN), a wide area network (WAN), awireless local area network (WLAN), a metropolitan area network (MAN), awide area network (WAN), a public telephone switched network (PSTN), aBluetooth™ network, a ZigBee network, a near field communication (NFC)network, or the like, or any combination thereof. In some embodiments,the network 120 may include one or more network access points. Forexample, the network 120 may include wired or wireless network accesspoints such as base stations and/or internet exchange points 120-1,120-2, . . . , through which one or more components of the strategyevaluation system 100 may be connected to the network 120 to exchangedata and/or information.

The storage device 130 may store data and/or instructions. In someembodiments, the storage device 130 may store data obtained from theuser terminal 140 and/or the processing engine 112. For example, thestorage device 130 may store behavior data of the one or more usersobtained from the user terminal 140. As another example, the storagedevice 130 may store a user group of a user determined by the processingengine 112. In some embodiments, the storage device 130 may store dataand/or instructions that the server 110 may execute or use to performexemplary methods described in the present disclosure. For example, thestorage device 130 may store instructions that the processing engine 112may execute or use to determine a value of a parameter in one or moredimensions regarding a user group (e.g., the first user group, thesecond user group). As another example, the storage device 130 may storeinstructions that the processing engine 112 may execute or use todetermine a parameter difference of the parameter regarding the firstuser group and the second user group. As still another example, thestorage device 130 may store instructions that the processing engine 112may execute or use to determine a reliability level of the parameterdifference. In some embodiments, the storage device 130 may include amass storage, a removable storage, a volatile read-and-write memory, aread-only memory (ROM), or the like, or any combination thereof.Exemplary mass storage may include a magnetic disk, an optical disk, asolid-state drive, etc. Exemplary removable storage may include a flashdrive, a floppy disk, an optical disk, a memory card, a zip disk, amagnetic tape, etc. Exemplary volatile read-and-write memory may includea random access memory (RAM). Exemplary RAM may include a dynamic RAM(DRAM), a double date rate synchronous dynamic RAM (DDR SDRAM), a staticRAM (SRAM), a thyrisor RAM (T-RAM), and a zero-capacitor RAM (Z-RAM),etc. Exemplary ROM may include a mask ROM (MROM), a programmable ROM(PROM), an erasable programmable ROM (EPROM), an electrically-erasableprogrammable ROM (EEPROM), a compact disk ROM (CD-ROM), and a digitalversatile disk ROM, etc. In some embodiments, the storage device 130 maybe implemented on a cloud platform. Merely by way of example, the cloudplatform may include a private cloud, a public cloud, a hybrid cloud, acommunity cloud, a distributed cloud, an inter-cloud, a multi-cloud, orthe like, or any combination thereof.

In some embodiments, the storage device 130 may be connected to thenetwork 120 to communicate with one or more components in the strategyevaluation system 100 (e.g., the server 110, the user terminal 140,etc.). One or more components in the strategy evaluation system 100 mayaccess the data or instructions stored in the storage device 130 via thenetwork 120. In some embodiments, the storage device 130 may be directlyconnected to or communicate with one or more components in the strategyevaluation system 100 (e.g., the server 110, the user terminal 140,etc.). In some embodiments, the storage device 130 may be part of theserver 110.

In some embodiments, the user terminal 140 may include a mobile device140-1, a tablet computer 140-2, a laptop computer 140-3, or the like, orany combination thereof. In some embodiments, the mobile device 140-1may include a smart home device, a wearable device, a mobile equipment,a virtual reality device, an augmented reality device, or the like, orany combination thereof. In some embodiments, the smart home device mayinclude a smart lighting device, a control device of an intelligentelectrical apparatus, a smart monitoring device, a smart television, asmart video camera, an interphone, or the like, or any combinationthereof. In some embodiments, the wearable device may include abracelet, footgear, glasses, a helmet, a watch, clothing, a backpack, asmart accessory, or the like, or any combination thereof. In someembodiments, the mobile equipment may include a mobile phone, a personaldigital assistance (PDA), a gaming device, a navigation device, a pointof sale (POS) device, a laptop, a desktop, or the like, or anycombination thereof. In some embodiments, the virtual reality deviceand/or the augmented reality device may include a virtual realityhelmet, a virtual reality glass, a virtual reality patch, an augmentedreality helmet, augmented reality glasses, an augmented reality patch,or the like, or any combination thereof. For example, the virtualreality device and/or the augmented reality device may include a GoogleGlass™, a RiftCon™, a Fragments™, a Gear VR™, etc.

It should be noted that the strategy evaluation system 100 is merelyprovided for the purposes of illustration, and is not intended to limitthe scope of the present disclosure. For persons having ordinary skillsin the art, multiple variations or modifications may be made under theteachings of the present disclosure. For example, the strategyevaluation system 100 may further include a database, an informationsource, or the like. As another example, the strategy evaluation system100 may be implemented on other devices to realize similar or differentfunctions. However, those variations and modifications do not departfrom the scope of the present disclosure.

FIG. 2 is a schematic diagram illustrating exemplary components of acomputing device on which the server 110, the storage device 130, and/orthe user terminal 140 may be implemented according to some embodimentsof the present disclosure. The computer device 200 may include aprocessor 210, a network interface 220, and a computer readable medium230. The computer device 200 may further include any other hardwareaccording to the actual function of the server 110. The processor 210may read and execute instructions associated with the strategyevaluation system 100 in the computer readable medium 230 to perform oneor more functions described in the present disclosure. The computerreadable medium 230 may include any electronic, magnetic, optical orphysical storage device that may contain or store information such asexecutable instructions and/or data. For example, the computer readablestorage medium 230 may include a random access memory (RAM), a volatilememory, a non-volatile memory, a flash memory, a storage drive (e.g., ahard disk drive), a solid state disk, a storage disk (e.g., CD, DVD,etc.), or the like, or any combination thereof.

The particular system may use a functional block diagram to explain thehardware platform containing one or more user interfaces. The computermay be a computer with general or specific functions. Both types of thecomputers may be configured to implement any particular system accordingto some embodiments of the present disclosure. Computing device 200 maybe configured to implement any components that perform one or morefunctions disclosed in the present disclosure. For example, thecomputing device 200 may implement any component of the strategyevaluation system 100 as described herein. In FIGS. 1-2, only one suchcomputer device is shown purely for convenience purposes. One ofordinary skill in the art would understood at the time of filing of thisapplication that the computer functions relating to the strategyevaluation as described herein may be implemented in a distributedfashion on a number of similar platforms, to distribute the processingload.

The computing device 200, for example, may also include COM portsconnected to and from a network connected thereto to facilitate datacommunications. The computing device 200 may include a processor (e.g.,the processor 210), in the form of one or more processors (e.g., logiccircuits), for executing program instructions. For example, theprocessor may include interface circuits and processing circuitstherein. The interface circuits may be configured to receive electronicsignals from a bus 240, wherein the electronic signals encode structureddata and/or instructions for the processing circuits to process. Theprocessing circuits may conduct logic calculations, and then determine aconclusion, a result, and/or an instruction encoded as electronicsignals. Then the interface circuits may send out the electronic signalsfrom the processing circuits via the bus 240.

The exemplary computing device may include the internal communicationbus 240, program storage and data storage of different forms including,for example, a disk, and a read only memory (ROM), or a random accessmemory (RAM), for various data files to be processed and/or transmittedby the computing device. The exemplary computing device may also includeprogram instructions stored in the ROM, RAM, and/or other type ofnon-transitory storage medium to be executed by the processor 210. Themethods and/or processes of the present disclosure may be implemented asthe program instructions. The computing device 200 may also include anI/O component, supporting input/output between the computer and othercomponents. The computing device 200 may also receive programming anddata via network communications.

Merely for illustration, only one CPU and/or processor is illustrated inFIG. 2. Multiple CPUs and/or processors are also contemplated; thusoperations and/or method steps performed by one CPU and/or processor asdescribed in the present disclosure may also be jointly or separatelyperformed by the multiple CPUs and/or processors. For example, if in thepresent disclosure the CPU and/or processor of the computing device 200executes both step A and step B, it should be understood that step A andstep B may also be performed by two different CPUs and/or processorsjointly or separately in the computing device 200 (e.g., the firstprocessor executes step A and the second processor executes step B, orthe first and second processors jointly execute steps A and B).

FIG. 3 is a schematic diagram illustrating exemplary hardware and/orsoftware components of an exemplary user terminal according to someembodiments of the present disclosure; on which the user terminal 140may be implemented according to some embodiments of the presentdisclosure. As illustrated in FIG. 3, the mobile device 300 may includea communication platform 310, a display 320, a graphic processing unit(GPU) 330, a central processing unit (CPU) 340, an I/O 350, a memory360, and a storage 390. The CPU 340 may include interface circuits andprocessing circuits similar to the processor 220. In some embodiments,any other suitable component, including but not limited to a system busor a controller (not shown), may also be included in the mobile device300. In some embodiments, a mobile operating system 370 (e.g., iOS™,Android™ Windows Phone™, etc.) and one or more applications 380 may beloaded into the memory 360 from the storage 390 in order to be executedby the CPU 340. The applications 380 may include a browser or any othersuitable mobile apps for receiving and rendering information relating toa service request or other information from the location based serviceproviding system on the mobile device 300. User interactions with theinformation stream may be achieved via the I/O devices 350 and providedto the processing engine 112 and/or other components of the strategyevaluation system 100 via the network 120.

In order to implement various modules, units and their functionsdescribed above, a computer hardware platform may be used as hardwareplatforms of one or more elements (e.g., a component of the sever 110described in FIG. 2). Since these hardware elements, operating systems,and program languages are common, it may be assumed that persons skilledin the art may be familiar with these techniques and they may be able toprovide information required in the route planning according to thetechniques described in the present disclosure. A computer with userinterface may be used as a personal computer (PC), or other types ofworkstations or terminal devices. After being properly programmed, acomputer with user interface may be used as a server. It may beconsidered that those skilled in the art may also be familiar with suchstructures, programs, or general operations of this type of computerdevice. Thus, extra explanations are not described for the figures.

FIG. 4 is a schematic diagram illustrating an exemplary processingengine according to some embodiments of the present disclosure. Theprocessing engine 112 may include a classification unit 410, anobtaining unit 420, an analysis unit 430, a transmission unit 440, ajudgment unit 450, and a response unit 460. The units may be hardwarecircuits of at least part of the processing engine 112. The units mayalso be implemented as an application or set of instructions read andexecuted by the processing engine 112. Further, the units may be anycombination of the hardware circuits and the application/instructions.For example, the units may be the part of the processing engine 112 whenthe processing engine 112 is executing the application/set ofinstructions.

The classification unit 410 may be configured to classify one or moreusers into a first user group and a second user group randomly. Thefirst user group may correspond to a first strategy, and the second usergroup may correspond to a second strategy.

In some embodiments, the first user group and the second user group maycorrespond to two strategies of a target project (e.g., two strategiesof a dynamical price adjustment project). In some embodiments, theclassification unit 410 may classify the one or more users into the usergroups using a random grouping algorithm. For example, theclassification unit 410 may classify the one or more users using asalted hash algorithm. As another example, the classification unit 410may assign a random number to each user, and classify the one or moreusers into two groups by comparing the random number with a presetthreshold.

The obtaining unit 420 may be configured to obtain data and/orinformation related to the strategy evaluation system 100. In someembodiments, the obtaining unit 420 may obtain a plurality of servicerequests from the one or more users. In some embodiments, the servicerequest may be a request for a service. In some embodiments, theobtaining unit 420 may obtain the behavior data of the one or moreusers. For example, the obtaining unit 420 may obtain the behavior dataof the one or more users from user logs. In some embodiments, theterminals of the one or more users (e.g., the user terminal 140) maygenerate user logs based on user activities on the application. The userlogs may include basic information of the user (e.g., user operation,user ID, the user group of the user, city of the user, gender of theuser, operation system of the terminal of the user) and interactions ofthe user with the user terminal 140.

In some embodiments, the obtaining unit 420 may obtain the data and/orinformation related to the strategy evaluation system 100 from a userterminal (e.g., the user terminal 140), the storage device 130, and/oran external data source (not shown). In some embodiments, the obtainingunit 420 may obtain the data and/or information related to the strategyevaluation system 100 via the network 120.

The analysis unit 430 may be configured to determine a first value of aparameter in one or multiple dimensions regarding the first user groupand a second value of the parameter in multiple dimensions regarding thesecond user group by analyzing the behavior data of the one or moreusers. In some embodiments, the analysis unit 430 may determine one ormore subgroups from the first user group and the second user group interms of the multiple dimensions, respectively. The analysis unit 430may determine one or more first values of a parameter in multipledimensions regarding the one or more subgroups in the first user groupand one or more second values of the parameter in multiple dimensionsregarding the one or more subgroups in the second user group based on abig data analysis method.

The analysis unit 430 may be configured to analyze information and/ordata related to the strategy evaluation system 100. In some embodiments,the analysis unit 430 may determine a parameter for evaluatingperformances of the two strategies. Merely by ways of example, theparameter may be associated with order information of a service. Takinga taxi hailing service as an example, the parameter may include thenumber of service orders, an order acceptance rate, an average responsetime of the service orders, or the like, or any combination thereof. Insome embodiments, the parameter may be in one or multiple dimensions. Insome embodiments, the analysis unit 430 may determine the first value ofthe parameter regarding the first user group and the second value of theparameter regarding the second user group. For example, the analysisunit 430 may determine the first value of the parameter in multipledimensions regarding the first user group and the second value of theparameter in multiple dimensions regarding the second user group basedon behavior data of the one or more users using a big data analysismethod. In some embodiments, the analysis unit 430 may select a usergroup from the first user group and the second user group. Merely by wayof example, the analysis unit 430 may select the user group from thefirst user group and the second user group by comparing the first valueand the second value. In some embodiments, the analysis unit 430 maydesignate a strategy corresponding to the selected user group as anoptimal strategy.

The transmission unit 440 may be configured to transmit the first valueand the second value to a visual interface of a terminal device. Forexample, the transmission unit 440 may transmit the first value and thesecond value to the visual interface of the terminal device. In someembodiments, the visual interface of the terminal device may display thefirst value and the second value in the form of text, graph, audio,video, or the like, or a combination thereof. The first value and thesecond value may be used to determine an optimal strategy from the firststrategy and the second strategy.

The judgment unit 450 may be configured to determine whether a userbelongs to any one of the first user group and the second user groupwhen the strategy evaluation system 100 obtains a service request fromthe user. If the user does not belong to a user group, theclassification unit 410 may classify the user into a user group. If theuser belongs to one of the first user group and the second user group,the response unit 460 may response the service request.

The response unit 460 may be configured to response the service requestfrom the user if the user belongs to one of the first user group and thesecond user group. Take a taxi-hailing service as an example, theresponse unit 460 may send the service request to a plurality of serviceproviders (e.g., a driver) that are available to accept the servicerequest.

It should be noted that the above description of the processing engine112 is merely provided for the purposes of illustration, and notintended to limit the scope of the present disclosure. For personshaving ordinary skills in the art, multiple variations and modificationsmay be made under the teachings of the present disclosure. For example,the processing engine 112 may further include a storage modulefacilitating data storage. As another example, the judgment unit 450and/or the response unit 460 may be omitted. However, those variationsand modifications do not depart from the scope of the presentdisclosure.

FIG. 5 is a schematic diagram illustrating an exemplary processingengine according to some embodiments of the present disclosure. In someembodiments, the processing engine 112 may include a parameter obtainingunit 510, a determination unit 520, and an output unit 530. The unitsmay be hardware circuits of at least part of the processing engine 112.The units may also be implemented as an application or set ofinstructions read and executed by the processing engine 112. Further,the units may be any combination of the hardware circuits and theapplication/instructions. For example, the units may be the part of theprocessing engine 112 when the processing engine 112 is executing theapplication/set of instructions.

The parameter obtaining unit 510 may be configured to obtain the firstvalue of a parameter regarding the first user group and the second valueof the parameter regarding the second user group. In some embodiments,the parameter obtaining unit 510 may obtain the first value and thesecond value from the analysis unit 430. In some embodiments, theparameter obtaining unit 510 may obtain the first value and the secondvalue from the storage device 130 and/or an external data source (notshown). The obtained first value and second value may be used toevaluate and/or adjust the optimization model.

The determination unit 520 may be configured to determine evaluationresults associated with the optimization model. In some embodiments, thedetermination unit 520 may determine a parameter difference of theparameter regarding the first user group and the second user group basedon the first value, the second value, and systematic errors of theoptimization model. In some embodiments, the parameter difference may bea difference value (also refer to as “first difference value”) or aratio value. Merely by ways of example, the first difference value maybe determined based on systematic errors of the optimization model (alsoreferred to as “second difference value”) and the difference between thefirst value and the second value (also referred to as “third differencevalue”). In some embodiments, the correction model may be the same as orsimilar to the optimization model except that the correction modelassociates the first user group and the second user group with a samestrategy. For example, in an A/A testing process, the first user groupand the second user group may correspond to the first strategy or thesecond strategy in the A/B testing process.

In some embodiments, the determination unit 520 may determine areliability level of the parameter difference. The reliability level ofthe parameter difference may refer to a repeatability of the parameterdifference when measurements are repeated a number of times. In someembodiments, the determination unit 520 may determine the reliabilitylevel based on a confidence interval of the parameter difference (e.g.,a first difference value) at a preset confidence coefficient. As anotherexample, the determination unit 520 may determine the reliability levelbased on a comparison of a P value of the parameter difference (e.g.,the first difference value) with a significance value. As still anotherexample, the determination unit 520 may determine the reliability levelbased on a confidence interval of the parameter difference (e.g., thedifference value, the ratio value) at a preset confidence coefficientand a P value of the parameter difference.

The output unit 530 may be configured to adjust the optimization model.In some embodiments, the output unit 530 may adjust the optimizationmodel based on the evaluation results associated with the optimizationmodel. In some embodiments, the output unit 530 may determine aplurality of parameter differences corresponding to a plurality ofparameters (e.g., the number of service orders, expenses on the service,user ratings, etc.) regarding the first user group and the second usergroup. In some embodiments, the output unit 530 may determine a finalscore relating to the optimization model based on parameter differencesand the reliability level of the parameter differences. As used herein,the final score relating to the optimization model maybe an evaluationresult of the optimization model in terms of the plurality ofparameters. In some embodiment, the final score relating to theoptimization model may be determined based on scores for the pluralityof parameter and weights of the plurality of parameters. In someembodiments, a weight of a parameter may indicate importance of theparameter in the evaluation of the optimization model.

In some embodiments, the output unit 530 may adjust the optimizationmodel based on the final score relating to the optimization model. Forexample, the output unit 530 may adjust the way that the one or moreusers are classified into the two groups. As another example, the outputunit 530 may adjust the way that the first value and the second value ofthe parameter are determined. Merely for illustration purposes, theadjustment of the optimization model may be an iterative processincluding one or more iterations. During each iteration, the output unit530 may adjust the optimization model based on the final score relatingto the optimization model. In some embodiments, the iterative processmay terminate when the final score is not less than a threshold. In someembodiments, the iterative process may terminate when a certain numberof iterations (e.g., 100 rounds, 300 rounds, etc.) is complete.

It should be noted that the above description of the processing engine112 is merely provided for the purposes of illustration, and notintended to limit the scope of the present disclosure. For personshaving ordinary skills in the art, multiple variations and modificationsmay be made under the teachings of the present disclosure. For example,the processing engine 112 may further include a storage unitfacilitating data storage. As another example, the parameter obtainingunit 510 may be omitted. However, those variations and modifications donot depart from the scope of the present disclosure.

FIG. 6 is a flow chart illustrating an exemplary process 600 fordetermining an optimal strategy according to some embodiments of thepresent disclosure. In some embodiments, the process 600 may beimplemented in the strategy evaluation system 100. For example, theprocess 600 may be stored in the storage device 130 and/or the storage(e.g., the computer readable medium 230) as a form of instructions, andinvoked and/or executed by the server 110 (e.g., the processing engine112 in the server 110, or the processor 210 of the processing engine 112in the server 110).

In 610, a plurality of service requests may be obtained from one or moreusers. The plurality of service requests may be obtained by, forexample, the obtaining unit 420. In some embodiments, the obtaining unit420 may obtain the service requests from the user terminal(s) 140 of theone or more users via the network 120.

In some embodiments, the user terminal 140 may establish a communication(e.g., wireless communication) with the server 110, for example, throughan application (e.g., the application 380 in FIG. 3) installed in theuser terminal 140. In some embodiments, the application may beassociated with a service (e.g., an online to offline service). Forexample, the application may be associated with a taxi-hailing service.In some embodiments, the user may log into the application, and initiatea service request by selecting one or more options on an interface ofthe application. In some embodiments, the application installed in theuser terminal 140 may direct the user terminal 140 to monitor servicerequests from the user continuously or periodically, and automaticallytransmit the service requests to the processing engine 112 via thenetwork 120.

In some embodiments, the service request may be a request for a service.Merely for illustration purposes, the service may include a taxiservice, a carpooling service, a hitch service, a delivery service, anonline shopping service, a party organization service, an unmanneddriving service, a medical service, a map-based service (e.g., a routeplanning service), a live chatting service, a query service, a sensorialexperience service, or the like, or any combination thereof. Take ataxi-hailing service as an example, the service request may include adeparture location, a destination, a start time, etc. The departurelocation may refer to a location where a requester starts his/herjourney. The destination may refer to a location where the requesterends his/her journey. The service request may further include a user'sidentity information (e.g., an identification (ID), a telephone number,a user's name, etc.).

In 620, the one or more users may be classified into a first user groupand a second user group using an optimization model. The one or moreusers may be classified by, for example, the classification unit 410. Insome embodiments, the optimization model (e.g., an A/B testing model)may be in forms of a collection of logic codes configured to performmultiple functions. For example, the optimization model may be used toclassify the one or more users into the first user group and the seconduser group. The first user group and the second user group maycorrespond to two strategies of a target project (e.g., two strategiesof a dynamical price adjustment project). The optimization model maycompare the two strategies and determine an optimal strategy from thetwo strategies. As used herein, a strategy may refer to a method or aplan to achieve the target project. The strategy may be visible orinvisible. For example, the strategy may include a certain design orfunction of an application in the user terminal(s) 140 of the one ormore users. As another example, the strategy may include a dynamic priceadjustment algorithm.

Merely for illustration purposes, the first strategy may be multiplyingthe price of a service by a coefficient 1 at peak hours (e.g.,8:00-9:00, 17:00-18:00, etc.), and the second strategy may bemultiplying the price by a coefficient 1.5 at peak hours. In someembodiments, the first user group may be a treatment group, and thesecond user group may be a control group. In some embodiments, the firstuser group may be the control group, and the second user group may bethe treatment group.

The classification unit 410 may classify the one or more users into twogroups when the strategy evaluation system 100 obtains service requestsfrom the one or more users. In some embodiments, the judgment unit 450may determine whether a user belongs to any one of the two user groupswhen the strategy evaluation system 100 obtains a service request fromthe user. In some embodiments, the judgment unit 450 may determinewhether the user belongs to a user group based on, for example, useridentity information in the service request. Upon a determination thatthe user belongs to a certain user group, the classification unit 510may classify the user into the user group. Upon a determination that theuser does not belong to a user group, the classification unit 410 mayclassify the user into a user group based on a random groupingalgorithm. For example, the classification unit 410 may classify the oneor more users using a salted hash algorithm. As another example, theclassification unit 410 may classify the one or more users based on arandom number assigned for each user and a preset threshold. Moredescriptions regarding the classification of a user into one of twogroups based on the random number and the preset threshold may be foundelsewhere in the present disclosure (e.g., FIG. 8 and the descriptionsthereof). After classifying the one or more users into the two usergroups, the processing engine 112 may store the user groups in a storagedevice (e.g., the storage device 130) of the strategy evaluation system100. The classification unit 410 may access the storage device andretrieve the user groups.

In some embodiments, after a user is classified into a user group, theresponse unit 460 may respond to the service request obtained from theuser. Take a taxi-hailing service as an example, the response unit 460may send the service request to a plurality of service providers (e.g.,a driver) that are available to accept the service request.

In 630, behavior data may be obtained from terminals of the one or moreusers associated with the two strategies. The behavior data may beobtained by, for example, the obtaining unit 420. In some embodiments,the terminals of the one or more users (e.g., the user terminal 140) maygenerate user logs based on user activities on the application. As usedherein, the user logs may refer to a set of files that record useractivities (e.g., select a service option in an application) when theuser operates the application. The user logs may include basicinformation of the user (e.g., user operation, user ID, the user groupof the user, city of the user, gender of the user, operation system ofthe terminal of the user) and the behavior data of the user. Merely forillustration purposes, the terminal of a user may generate a user log“JASON: {“event_id”: “fast_order_click”; “passenger_id”: “115116”;“test_group”: “treatment”; “city”: “beijing”; “gender”: “man”;“system_type”: “iOS”}”, where “event_id” may refer to an user operation,“fast_order_click” may refer that the user operation is requesting anorder, “passenger_id” may refer to a user ID, “115116” may refer thatthe user ID is 115116, “test_group” may refer to the user group that theuser belongs to, “treatment” may refer to that the user belongs to thetreatment group, “city” may refer to the city where the user is,“beijing” may refer to that the user is in Beijing, “gender” may referto the gender of the user, “man” may refer to that the gender of theuser is man, “system_type” may refer to the type of the operation systemof the terminal of the user, “iOS” may refer to that the operationsystem of the terminal of the user is iOS.

The obtaining unit 420 may continuously or periodically obtain the userlogs from the terminals of the one or more users. In some embodiments,the terminals of the one or more users may transmit the user logs to thestorage device (e.g., the storage device 130) via the network 120continuously or periodically. The obtaining unit 420 may access thestorage device, and retrieve the user logs. In some embodiments, theobtaining unit 420 may obtain the user logs using a Hadoop DistributeFile System.

In 640, a first value of a parameter regarding the first user group anda second value of the parameter regarding the second user group may bedetermined based on behavior data of the one or more users using theoptimization model. The first value and the second value may bedetermined by, for example, the analysis unit 430. The parameter mayindicate user feedbacks from the one or more users on the first strategyand the second strategy. In some embodiments, the parameter may reflectthe user preferences for the first strategy and the second strategy.Merely by ways of example, the parameter may be associated with orderinformation of an online to offline service. Taking a taxi hailingservice as an example, the parameter may include the number of serviceorders, an order acceptance rate, an average response time of theservice orders, or the like, or any combination thereof. In someembodiments, the parameter may be in multiple dimensions. In someembodiments, the multiple dimensions may be represented by multipleattributes of the user. For example, the multiple dimensions may includethe age of the user, the gender of the user, the city where the userlives, the operation system of the terminal of the user, or the like, orany combination thereof. Merely for illustration purposes, a parameter“average number of service orders” in multiple dimensions like“Beijing”, “man”, and “iOS” may refer that the average number of serviceorders for men in Beijing using a terminal with an iOS operation system.In some embodiments, the parameter and/or the multiple dimensions of theparameters may be selected according to scenario applications. Moredescriptions regarding the multiple dimensions may be found elsewhere inthe present disclosure (e.g. FIG. 7B and the descriptions thereof).

In some embodiments, the analysis unit 430 may determine the first valueof the parameter in one or multiple dimensions regarding the first usergroup and the second value of the parameter in one or multipledimensions regarding the second user group based on the behavior datausing a big data analysis method. Taking a taxi hailing service as anexample, the parameter may be “average number of service orders”, andthe dimension may be “Beijing”. The analysis unit 430 may determine asubgroup based on the behavior data of the users in the first group andthe dimension. The users in the subgroup may be people who live inBeijing in the first user group. The analysis unit 430 may furtherdetermine the average number of service orders (i.e., the first value)of the users in the subgroup based on behavior data of the users.Similarly, the analysis unit 430 may determine the second value based onthe behavior data of users in the second user group. As another example,the parameter may be “average number of service orders”, and themultiple dimensions may be “Beijing”, “man”, and “iOS”. The analysisunit 430 may determine a subgroup based on the behavior data of theusers in the first user group and the multiple dimensions. The users inthe subgroup may select men in Beijing who use IOS operation systems intheir terminals from the first user group. The analysis unit 430 mayfurther determine the average number of service orders (i.e., the firstvalue) of the users in the subgroup based on behavior data of the users.Similarly, the analysis unit 430 may determine the second value based onthe behavior data of users in the second user group.

In 650, the first value and the second value may be transmitted to aterminal device. The first value and the second value may be transmittedby, for example, the transmission unit 440. The terminal device hereinmay be used to display information associated with the strategyevaluation system 100 (e.g., the first value, the second value) to auser (e.g., a technician, a decision maker associated with the twostrategies). In some embodiments, the transmission unit 440 may transmitthe first value and the second value to a visual interface of theterminal device via the network 120. The visual interface of theterminal device may display the first value and the second value in theform of text, graph, audio, video, or the like, or any combinationthereof.

In 660, a user group may be selected from the first user group and thesecond user group based on the first value and the second value. Theuser group may be selected by, for example, the analysis unit 430. Insome embodiments, the analysis unit 430 may select the user group fromthe first user group and the second user group by comparing the firstvalue and the second value. Merely for illustration purposes, if theparameter is “the number of service orders”, the analysis unit 430 maydesignate a user group with a greater number of service orders as aselected user group.

In 670, a strategy corresponding to the selected user group may bedetermined as an optimal strategy. The optimal strategy may bedetermined by, for example, the analysis unit 430. The optimal strategymay be applied to the target project. For example, an optimal strategyof multiplying the price of a service by a coefficient 1 at peak hours(e.g., 8:00-9:00, 17:00-18:00, etc.) may be applied to a dynamical priceadjustment project in a taxi-hailing service.

It should be noted that the above description is merely provided for thepurposes of illustration, and not intended to limit the scope of thepresent disclosure. For persons having ordinary skills in the art,multiple variations and modifications may be made under the teachings ofthe present disclosure. In some embodiments, process 600 may beperformed multiple times to determine an optimal strategy from aplurality of strategies. In some embodiments, one or more steps may beadded or omitted. For example, step 650 may be omitted. As anotherexample, step 660 and step 670 may be integrated into a single step.However, those variations and modifications do not depart from the scopeof the present disclosure.

FIG. 7A is a schematic diagram illustrating an exemplary process fordetermining an optimal strategy according to some embodiments of thepresent disclosure. In some embodiments, the process 700 may illustratethe process for determining an optimal strategy in combination with theprocess 600 in FIG. 6. As shown in FIG. 7, a plurality of servicerequests may be obtained from a plurality of users in 710, and theplurality of users may be classified into a first user group and asecond user group in 720. For example, the plurality of users may beclassified into the two user groups according to a classificationalgorithm, such as a piecewise function, a machine learning model. Thefirst user group and the second user group may correspond to twostrategies, respectively. Behavior data of the plurality of usersassociated with the two strategies may be obtained from terminals of theplurality of users in 730. Then a first value and a second value of aparameter in multiple dimensions (e.g., the number of service orders ofmen in Beijing in a month) regarding the first user group and the seconduser group may be determined, respectively. The first value and thesecond value may be compared in forms of, for example, a diagram in 740.Finally, an optimal strategy may be determined from the two strategiesbased on the comparison of the first value and the second value.

FIG. 7B is a schematic diagram illustrating an exemplary parameter inmultiple dimensions according to some embodiments of the presentdisclosure. As illustrated in FIG. 7B, three axes X, Y, and Z refer tothree dimensions of a parameter M (e.g., number of service orders,negative feedbacks of the service orders, etc.). For example, the axis Xmay refer to the city where a user lives, the axis Y may refer to thegender of the user, and the axis Z may refer to the education backgroundof the user. M_(A) may refer to a value of the parameter regarding afirst user group. M_(B) may refer to a value of the parameter regardinga second user group. M_(A) (Y_(n), Z_(n)) and M_(A)(X_(n), Y_(n), Z_(n))may refer to first values of the parameter M in two dimensions (i.e.,dimensions Y and Z) and in three dimensions (i.e., dimensions X, Y, andZ) regarding the first user group, respectively. M_(B) (Y_(n), Z_(n))and M_(B) (X_(n), Y_(n), Z_(n)) may refer to second values of theparameter M in two dimensions (i.e., dimensions Y and Z) and in threedimensions (i.e., dimensions X, Y, and Z) regarding the second usergroup, respectively. Merely by way of example, M_(A)(X_(n), Y_(n),Z_(n)) may refer to the number of service orders from men in Beijingwhose education background is undergraduate.

FIG. 8 is a flow chart illustrating an exemplary process 800 forclassifying one or more users into a user group according to someembodiments of the present disclosure. In some embodiments, the process800 may be implemented in the strategy evaluation system 100. Forexample, the process 800 may be stored in the storage device 130 and/orthe storage (e.g., the computer readable medium 230) as a form ofinstructions, and invoked and/or executed by the server 110 (e.g., theprocessing engine 112 in the server 110, or the processor 210 of theprocessing engine 112 in the server 110). In some embodiments, one ormore operations in the process 800 may be performed by theclassification unit 410.

In 810, a random number for each of the one or more users may begenerated. In some embodiments, the random number may be any positivenumbers, for example, 0.8, 1, 5, 12, 88, etc. In some embodiments, therandom numbers may be in a certain range, for example, 1˜100.

In 820, a determination may be made as to whether the random number fora user is greater than a threshold. The threshold may be set by a user(e.g., a technician, a decision maker associated with the twostrategies), according to default settings of the strategy evaluationsystem 100, or adjusted under different situations. In some embodiments,the threshold may be a predetermined number of users in the first usergroup. In some embodiments, the threshold may be a value thatcorresponds to a predetermined percentage (e.g., 40%, 60%, or 80%) ofthe users in the first user group in terms of all of the plurality ofthe users. For example, if percentages of the users in the first usergroup and the second user group are 60% and 40%, respectively, thethreshold may be set as 60.

If the random number of the user is greater than the threshold, theprocess 800 may proceed to 830. In 830, the classification unit 410 mayclassify the user into the first user group.

If the random number of the user is not greater than the threshold, theprocess 800 may proceed to 840. In 840, the classification unit 410 mayclassify the user into the second user group.

FIG. 9 is a flow chart illustrating an exemplary process 900 foradjusting an optimization model according to some embodiments of thepresent disclosure. In some embodiments, the process 900 may beimplemented in the strategy evaluation system 100. For example, theprocess 900 may be stored in the storage device 130 and/or the storage(e.g., the computer readable medium 230) as a form of instructions, andinvoked and/or executed by the server 110 (e.g., the processing engine112 in the server 110, or the processor 210 of the processing engine 112in the server 110). In some embodiments, the operations in the process900 may be performed by the determination unit 520 and/or the outputunit 530.

In 910, a parameter difference of the parameter regarding the first usergroup and the second user group may be determined. In some embodiments,the parameter difference may represent a difference between the firstvalue of the parameter regarding the first user group and the secondvalue of the parameter regarding the second user group if theoptimization model has no systematic errors. In some embodiments, thesystematic errors of the optimization model may relate to algorithmsused in the optimization model. In some embodiments, the parameterdifference may be a difference value (also refer to as “first differencevalue”) or a ratio value. In some embodiments, if the systematic errorsof the optimization model are considered, the first difference value maybe determined based on systematic errors of the optimization model (alsoreferred to as “second difference value”) and the difference between thefirst value of the parameter regarding the first user group and thesecond value of the parameter regarding the second user group (alsoreferred to as “third difference value”). For example, the firstdifference value may be determined by subtracting the second differencevalue from the third difference. In some embodiments, the seconddifference value may be determined using a correction model (e.g., anA/A testing model).

In some embodiments, if the systematic errors of the optimization modelare considered, the ratio value may be determined based on systematicerrors of the optimization model and a ratio between the first value andthe second value. For example, the ratio value may be determined bydividing the second value by the first value, then multiplying thequotient of the second value and the first value by a correctioncoefficient associated with the systematic errors of the optimizationmodel.

In 920, a reliability level of the parameter difference may bedetermined. As used herein, the reliability level of the parameterdifference may refer to a repeatability of the parameter difference whenmeasurements are repeated for a number of times. The determination unit520 may determine the reliability level in various ways. In someembodiments, the determination unit 520 may determine the reliabilitylevel based on a confidence interval of the parameter difference (e.g.,the first difference value) at a preset confidence coefficient. Theconfidence interval of the parameter difference at the preset confidencecoefficient may refer to that the probability that parameter difference(e.g., the first difference value) falls in the confidence interval isassociated with the preset confidence coefficient. Merely forillustration purpose, the confidence interval (e.g., (50, 80)) of theparameter difference at a preset confidence coefficient (e.g., 95%) mayrefer to the probability that the parameter difference falls in therange of (50, 80) is 95%. The preset confidence coefficient may be setmanually by a user, or determined by one or more components of thestrategic evaluation system 100 according to default settings. Forexample, the preset confidence coefficient may be 90%, 95%, or 99%. Moredescriptions regarding the determination of the reliability level basedon the confidence interval of the difference value at the presetconfidence coefficient may be found elsewhere in the present disclosure(e.g., FIG. 12 and the descriptions thereof).

In some embodiments, the determination unit 520 may determine thereliability level based on a comparison of a P value of the parameterdifference (e.g., the first difference value) with a significance value.The comparison of the P value of the difference value with thesignificance value may be referred to as a hypothesis testing, which maybe used to evaluate a null hypothesis and an alternative hypothesisabout the parameter difference. The null hypothesis and the alternativehypothesis may be proposed by a user, or one or more components of thestrategic evaluation system 100 according to default settings. Merely byway of example, the null hypothesis may be that the first value of theparameter regarding the first user group is the same as the second valueof the parameter regarding the second user group (i.e., the thirddifference value is 0). The alternative hypothesis may be that the firstvalue of the parameter regarding the first user group is different fromthe second value of the parameter regarding the second user group (i.e.,the third difference value is not 0).

In some embodiments, the significance value may be set manually by auser, or determined by one or more components of the strategicevaluation system 100 according to default settings. For example, thesignificance value may be 0.01, 0.05, or 0.10. Merely for illustrationpurpose, if the P value is less than or equal to the significance value(e.g., P≤0.05), the null hypothesis may be rejected, and the alternativehypothesis may be accepted. If the P value is greater than thesignificance value (e.g., P>0.05), the null hypothesis may be accepted.More descriptions of the determination of the reliability level based onthe comparison of the P value of the first difference value with thesignificance value may be found elsewhere in the present disclosure(e.g., FIG. 13 and the descriptions thereof).

In some embodiments, the determination unit 520 may determine thereliability level based on a confidence interval of the parameterdifference (e.g., the difference value, the ratio value) at a presetconfidence coefficient and a P value of the parameter difference. Forexample, the determination unit 520 may determine the confidenceinterval of the ratio value at the preset confidence coefficient, thencompare a P value of the ratio value with a significance value.

In 930, a final score relating to the optimization model may bedetermined based on the parameter difference and the reliability levelof the parameter difference. In some embodiments, the output unit 530may determine a plurality of parameter differences corresponding to aplurality of parameters (e.g., the number of service orders, expenses onthe service, user ratings, etc.) regarding the first user group and thesecond user group. The output unit 530 may determine a plurality ofreliability levels of the plurality of parameter differences. The outputunit 530 may determine a score relating to the optimization model foreach parameter based on the parameter difference of the parameter andthe reliability level of the parameter difference. As used herein, ascore for a parameter may be an evaluation result of the optimizationmodel in terms of the parameter. The output unit 530 may furtherdetermine a final score relating to the optimization model based on thescores for the plurality of parameter and weights of the plurality ofparameters. As used herein, the final score may be an evaluation resultof the optimization model in terms of the plurality of parameters. Theweight of a parameter may indicate importance of the parameter in theevaluation of the optimization model. Merely for illustration purposes,a first score for a first parameter, a second score for a secondparameter, and a third score for a third parameter are 80, 90, and 95,respectively, and a first weight of the first parameter, a second weightof the second parameter, and a third weight of the third parameter are20%, 30%, and 50%, respectively, the final score may be 90.5(80×20%+90×30%+95×50%=90.5).

In 940, the optimization model may be adjusted based on the final scorerelating to the optimization model. In some embodiments, the output unit530 may adjust the optimization model if the final score is less than athreshold. For example, the output unit 530 may adjust the way that theone or more users are classified into the two groups. As anotherexample, the output unit 530 may adjust the way that the first value andthe second value of the parameter are determined. In some embodiments,the adjustment of the optimization model may be an iterative processincluding one or more iterations. During each iteration, the output unit530 may adjust the optimization model based on the final score relatingto the optimization model. In some embodiments, the iterative processmay terminate when the final score is not less than the threshold. Insome embodiments, the iterative process may terminate when a certainnumber of iterations (e.g., 100 rounds, 300 rounds, etc.) is complete.

It should be noted that the above description is merely provided for thepurposes of illustration, and not intended to limit the scope of thepresent disclosure. For persons having ordinary skills in the art,multiple variations and modifications may be made under the teachings ofthe present disclosure. In some embodiments, one or more steps may beadded or omitted. For example, step 930 and step 940 may be integratedinto a single step. However, those variations and modifications do notdepart from the scope of the present disclosure.

FIG. 10 is a flow chart illustrating an exemplary process 1000 fordetermining a parameter difference of a parameter according to someembodiments of the present disclosure. In some embodiments, the process1000 may be implemented in the strategy evaluation system 100. Forexample, the process 1000 may be stored in the storage device 130 and/orthe storage (e.g., the computer readable medium 230) as a form ofinstructions, and invoked and/or executed by the server 110 (e.g., theprocessing engine 112 in the server 110, or the processor 210 of theprocessing engine 112 in the server 110). In some embodiments, theoperations in the process 1000 may be performed by the determinationunit 520.

In 1010, a second difference value may be determined. The seconddifference value may refer to the systematic errors of the optimizationmodel. In some embodiments, the determination unit 520 may determine thesecond difference value based on a third value of the parameterregarding the first user group and a fourth value of the parameterregarding the second user group. The third value and the fourth valuemay be determined using a correction model (e.g., an A/A testing model).For example, the second difference value may be determined bysubtracting the fourth value from the third value. More descriptions ofthe determination of the second difference value may be found elsewherein the present disclosure (e.g., FIG. 11 and the descriptions thereof).

In 1020, a third difference value may be determined based on the firstvalue and the second value. In some embodiments, the third differencevalue may be a difference between the first value and the second value.For example, the determination unit 520 may determine the thirddifference value by subtracting the first value (or the second value)from the second value (or the first value).

In 1030, a first difference value may be determined based on the seconddifference value and the third difference value. In some embodiments,the determination unit 520 may determine the first difference value bysubtracting the second difference value from the third difference value,thus eliminating or reducing the systems errors. For example, the seconddifference value is 0.4 and the third difference value is 1.2. The firstdifference value may be 0.8 (1.2−0.4=0.8). In some embodiments, thedetermination unit 520 may determine the first difference value using aCausalImpact model. In some embodiments, the CausalImpact model mayinclude a structural Bayesian time-series model to estimate causaleffect of a designed intervention on a time series.

In 1040, the first difference value may be designated as the parameterdifference of the parameter regarding the first user group and thesecond user group.

It should be noted that the above description is merely provided for thepurposes of illustration, and not intended to limit the scope of thepresent disclosure. For persons having ordinary skills in the art,multiple variations and modifications may be made under the teachings ofthe present disclosure. For example, steps 1010 and 1020 may be mergedinto one step. As another example, steps 1010 and 1020 may be performedsimultaneously or in any order. However, those variations andmodifications do not depart from the scope of the present disclosure.

FIG. 11 is a flow chart illustrating an exemplary process 1100 fordetermining a second difference value according to some embodiments ofthe present disclosure. In some embodiments, the process 1100 may beimplemented in the strategy evaluation system 100. For example, theprocess 1100 may be stored in the storage device 130 and/or the storage(e.g., the computer readable medium 230) as a form of instructions, andinvoked and/or executed by the server 110 (e.g., the processing engine112 in the server 110, or the processor 210 of the processing engine 112in the server 110). In some embodiments, the operations in the process1100 may be performed by the determination unit 520.

In 1110, behavior data of the one or more users associated with one ofthe two strategies may be obtained. In some embodiments, the behaviordata of the one or more users may be obtained from the user logs of theone or more users. In some embodiments, the operation for obtainingbehavior data of the one or more users may be the same as or similar tothe operations in 630. The one or more users may correspond to a samestrategy. For example, the one or more users may correspond to astrategy of multiplying the price of a service by a coefficient 1 atpeak hours (e.g., 8:00-9:00, 17:00-18:00, etc.).

In 1120, a third value of the parameter regarding the first user groupand a fourth value of the parameter regarding the second user group maybe determined based on the behavior data of the one or more usersassociated with the one of the two strategies using a correction model.As used herein, the correction model (e.g., the A/A testing model) maybe configured to correct systematic errors of the optimization model. Insome embodiments, the correction model may be the same as or similar tothe optimization model except that the correction model associates thefirst user group and the second user group with a same strategy. Forexample, in an A/A testing process, the first user group and the seconduser group may correspond to the first strategy or the second strategy.

In 1130, the second difference value may be determined based on thethird values and the fourth values. In some embodiments, the seconddifference value may be a difference between the third value and thefourth value. For example, the determination unit 520 may determine thesecond difference value by subtracting the third value (or the fourthvalue) from the fourth value (or the third value). In some embodiments,if the second difference value is 0, it may indicate that there is nosystematic errors in the optimization model.

In some embodiments, the determination unit 520 may determine thereliability level of the second difference value based on a comparisonof the P value with a significance value. More descriptions of thedetermination of the P value may be found elsewhere in the presentdisclosure (e.g., FIG. 13 and the descriptions thereof). In someembodiments, if the P value is not greater than the significance value(e.g., P≤0.05), it may indicate that the parameter difference of theparameter regarding the first user group and the second user grouprelating to the correction model is reliable. In this case, thedetermination unit 520 may stop using the optimization model, and designa new optimization model. If the p value is greater than thesignificance value (e.g., P>0.05), it may indicate that the parameterdifference of the parameter regarding the first user group and thesecond user group relating to the correction model is not reliable,i.e., the parameter difference is not significant. In this case, thedetermination unit 520 may correct the optimization model using thecorrection model.

It should be noted that the above description is merely provided for thepurposes of illustration, and not intended to limit the scope of thepresent disclosure. For persons having ordinary skills in the art,multiple variations and modifications may be made under the teachings ofthe present disclosure. For example, when determining the optimalstrategy, the process 1100 may be performed before the process 600, orthe process 1100 and the process 600 may be performed at the same time.However, those variations and modifications do not depart from the scopeof the present disclosure.

FIG. 12 is a flow chart illustrating an exemplary process 1200 fordetermining a reliability level of the parameter difference according tosome embodiments of the present disclosure. In some embodiments, theprocess 1200 may be implemented in the strategy evaluation system 100.For example, the process 1200 may be stored in the storage device 130and/or the storage (e.g., the computer readable medium 230) as a form ofinstructions, and invoked and/or executed by the server 110 (e.g., theprocessing engine 112 in the server 110, or the processor 210 of theprocessing engine 112 in the server 110). In some embodiments, theoperations in the process 1200 may be performed by the determinationunit 520.

In 1201, a confidence interval of the first difference value at a presetconfidence coefficient may be determined. The confidence interval of thefirst difference value at the preset confidence coefficient may refer tothat the probability that the first difference value falls in theconfidence interval is associated with the preset confidencecoefficient. Merely for illustration purpose, the confidence interval(e.g., (50, 90)) at the preset confidence coefficient (e.g., 95%) mayrefer to the probability that the first difference value falls in therange of (50, 90) is 95%.

In some embodiments, the parameter may be an average number of serviceorders. Merely for illustration purposes, the number of service ordersfor each user in the first user group may be represented as (x₁, x₂, x₃,. . . , x_(n1)), where n₁ refers to the number of users in the firstuser group, x_(i) refers to the number of service orders for an i-thuser in the first user group. In some embodiments, the average number ofservice orders in the first user group may be determined according toEquation (1):

$\begin{matrix}{{m_{1} = {\sum\limits_{i = 1}^{n_{1}}\frac{x_{i}}{n_{1}}}},} & (1)\end{matrix}$where m₁ refers to the average number of service orders in the firstuser group, x_(i) refers to the number of service orders for an i-thuser in the first user group, and n₁ refers to the number of users inthe first user group. A variance of the number of service orders in thefirst user group may be determined according to Equation (2):

$\begin{matrix}{{\sigma_{1} = {\sum\limits_{i = 1}^{n_{1}}\frac{\left( {x_{i} - m_{i}} \right)^{2}}{\left( {n_{1} - 1} \right)}}},} & (2)\end{matrix}$where σ1 refers to the variance of the number of service orders in thefirst user group, x_(i) refers to the number of service orders for ani-th user in the first user group, m₁ refers to the average number ofservice orders in the first user group, and n₁ refers to the number ofusers in the first user group. Similarly, it is assumed that the numberof service orders for each user in the second user group may berepresented as (y₁, y₂, y₃, . . . , y_(n2)), where n₂ refers to thenumber of users in the second user group, y_(i) refers to the number ofservice orders for an i-th user in the second user group. In someembodiments, the average number of service orders in the first usergroup may be determined according to Equation (3):

$\begin{matrix}{{m_{2} = {\sum\limits_{i = 1}^{n_{2}}\frac{y_{i}}{n_{2}}}},} & (3)\end{matrix}$where m₂ refers to the average number of service orders in the seconduser group, y_(i) refers to the number of service orders for an i-thuser in the first user group, and n₂ refers to the number of users inthe second user group. A variance of the number of service orders in thesecond user group may be determined according to Equation (4):

$\begin{matrix}{{\sigma_{2} = {\sum\limits_{i = 1}^{n_{2}}\frac{\left( {y_{i} - m_{2}} \right)^{2}}{\left( {n_{2} - 1} \right)}}},} & (4)\end{matrix}$where σ₂ refers to the variance of the number of service orders in thesecond user group, yi refers to the number of service orders for an i-thuser in the second user group, m₂ refers to the average number ofservice orders in the second user group, and n₂ refers to the number ofusers in the second user group. In some embodiments, the firstdifference value of the parameter regarding the first user group and thesecond user group may be determined according to Equation (5):D ₁ =m ₂ −m ₁ ±Δu  (5)where D₁ refers to the first difference value, m₁ refers to the averagenumber of service orders in the first user group, m₂ refers to theaverage number of service orders in the second user group, and Δu refersto the second difference value (i.e., the systematic errors of theoptimization model).

In some embodiments, a start point of the confidence interval may bedetermined according to Equation (6):

$\begin{matrix}{{C_{1} = {D_{1} = {Z_{1 - \frac{\alpha}{2}}*\sqrt{\frac{\sigma_{1}}{n_{1}} + \frac{\sigma_{2}}{n_{2}}}}}},} & (6)\end{matrix}$where C₁ refers to the start point of the confidence interval, D₁ refersto the first difference value, σ₁ refers to the variance of the numberof service orders in the first user group, σ₂ refers to the variance ofthe number of service orders in the second user group, n₁ refers to thenumber of users in the first user group, n₂ refers to the number ofusers in the second user group, α refers to a significance value, 1−αrefers to the confidence coefficient, and

$Z_{1 - \frac{\alpha}{2}}$may be determined based on a distribution of the parameter in math. Insome embodiments, the obtaining unit 420 may obtain the distribution ofthe parameter by consulting a distribution table. In some embodiments,the distribution of the parameter in math may be determined according tothe type of the parameter. For example, if the parameter is an averagenumber of service orders, the distribution of the parameter in math maybe a Bernoulli distribution. As another example, if the parameter isexpense of users, the distribution of the parameter in math may be anormal distribution. An end point of the confidence interval may bedetermined according to Equation (7):

$\begin{matrix}{{C_{2} = {D_{1} + {Z_{1 - \frac{\alpha}{2}}*\sqrt{\frac{\sigma_{1}}{n_{1}} + \frac{\sigma_{2}}{n_{2}}}}}},} & (7)\end{matrix}$where C₂ refers to the end point of the confidence interval, D₁ refersto the first difference value, σ₁ refers to the variance of the numberof service orders in the first user group, σ₂ refers to the variance ofthe number of service orders in the second user group, n₁ refers to thenumber of users in the first user group, n₂ refers to the number ofusers in the second user group, a refers to the significance value, 1−αrefers to the confidence coefficient, and

$Z_{1 - \frac{\alpha}{2}}$may be determined based on the distribution of the parameter in math.Accordingly, the confidence interval of the difference value at thepresent confidence coefficient may be determined according to Equation(8):

$\begin{matrix}{\left( {{{D\; 1} - {Z_{1 - \frac{\alpha}{2}}*\sqrt{\frac{\sigma 1}{n\; 1} + \frac{\sigma 2}{n\; 2}}}},{{D\; 1} + {Z_{1 - \frac{\alpha}{2}}*\sqrt{\frac{\sigma 1}{n\; 1} + \frac{\sigma 2}{n\; 2}}}}} \right).} & (8)\end{matrix}$

In 1220, the reliability level of the parameter difference may bedetermined based on the confidence interval of the difference value atthe preset confidence coefficient. In some embodiments, if theconfidence interval includes 0, it may indicate that the parameterdifference of the parameter (e.g., the average number of service orders)regarding the first user group and the second user group is notsignificant. If the confidence interval does not include 0, and both thestart point and the end point of the confidence interval are greaterthan 0, it may indicate that the average number of service orders in thesecond user group associated with the second strategy is greater thanthe number of service orders in the first user group associated with thefirst strategy. If the confidence interval does not include 0, and boththe start point and the end point of the confidence interval are lessthan 0, it may indicate that the average number of service orders in thesecond user group associated with the second strategy is less than theaverage number of service orders in the first user group associated withthe first strategy.

FIG. 13 is a flow chart illustrating an exemplary process 1300 fordetermining a reliability level of the parameter difference according tosome embodiments of the present disclosure. In some embodiments, theprocess 1300 may be implemented in the strategy evaluation system 100.For example, the process 1300 may be stored in the storage device 130and/or the storage (e.g., the computer readable medium 230) as a form ofinstructions, and invoked and/or executed by the server 110 (e.g., theprocessing engine 112 in the server 110, or the processor 210 of theprocessing engine 112 in the server 110). In some embodiments, theoperations in the process 1300 may be performed by the determinationunit 520.

In 1310, a P value may be determined based on the first difference valueand a distribution of the parameter in math. In some embodiments, theparameter may be an average number of service orders. Merely forillustration purposes, a statistics value may be determined according toEquation (9):

$\begin{matrix}{{t = {D_{1}/\sqrt{\frac{\sigma 1}{n\; 1} + \frac{\sigma 2}{n\; 2}}}},} & (9)\end{matrix}$where t refers to the statistics value, D₁ refers to the firstdifference value (e.g., determined according to Equation (5)), σ₁ refersto the variance of the number of service orders in the first user group(e.g., determined according to Equation (2)), σ₂ refers to the varianceof the number of service orders in the second user group (e.g.,determined according to Equation (4)), n1 refers to the number of usersin the first user group; n2 refers to the number of users in the seconduser group. The statistics value may be, for example, standardized valuethat is determined based on obtained data (e.g., the behavior data ofthe users) during the hypothesis test. The P value may be determinedaccording to Equation (10):P=2·p(z>|t|)  (10)where P refers to the P value; p(z>∛t|) refers to an area enclosed bythe normal distribution curve and the abscissa in a range where theaverage number of service orders is greater than t, and t refers to thestatistics value.

In 1320, the P value may be compared with a significance value. In someembodiments, the significance value may be set manually by a user, or bedetermined by one or more components of the strategic evaluation system100 according to default settings. For example, the significance valuemay be 0.05.

In 1330, the reliability level of the parameter difference may bedetermined based on the comparison of the P value with the significancevalue. In some embodiments, a null hypothesis and an alternativehypothesis may be proposed by a user, or one or more components of thestrategic evaluation system 100 according to default settings. Forexample, the null hypothesis may be that the first value of theparameter regarding the first user group is the same as the second valueof the parameter regarding the second user group (i.e., the firstdifference value is 0). The alternative hypothesis may be that the firstvalue of the parameter regarding the first user group is different fromthe second value of the parameter regarding the second user group (i.e.,the first difference value is not 0). In some embodiments, the nullhypothesis and the alternative hypothesis may be interchangeable. Insome embodiments, if the P value is not greater than the significancevalue (e.g., P≤0.05), the null hypothesis may be rejected, whichindicates that the first value of the parameter regarding the first usergroup is different from the second value of the parameter regarding thesecond user group. In this case, a smaller P value may correspond to ahigher reliability level of the parameter difference. If the P value isgreater than the significance value (e.g., P>0.05), the null hypothesismay be accepted, which indicates that, the first value of the parameterregarding the first user group is the same as the second value of theparameter regarding the second user group. In this case, the parameterdifference of the parameter may not be reliable.

Having thus described the basic concepts, it may be rather apparent tothose skilled in the art after reading this detailed disclosure that theforegoing detailed disclosure is intended to be presented by way ofexample only and is not limiting. Various alterations, improvements, andmodifications may occur and are intended to those skilled in the art,though not expressly stated herein. These alterations, improvements, andmodifications are intended to be suggested by this disclosure, and arewithin the spirit and scope of the exemplary embodiments of thisdisclosure.

Moreover, certain terminology has been used to describe embodiments ofthe present disclosure. For example, the terms “one embodiment,” “anembodiment,” and “some embodiments” mean that a particular feature,structure or characteristic described in connection with the embodimentis included in at least one embodiment of the present disclosure.Therefore, it is emphasized and should be appreciated that two or morereferences to “an embodiment” or “one embodiment” or “an alternativeembodiment” in various portions of this specification are notnecessarily all referring to the same embodiment. Furthermore, theparticular features, structures or characteristics may be combined assuitable in one or more embodiments of the present disclosure.

Further, it will be appreciated by one skilled in the art, aspects ofthe present disclosure may be illustrated and described herein in any ofa number of patentable classes or context including any new and usefulprocess, machine, manufacture, or composition of matter, or any new anduseful improvement thereof. Accordingly, aspects of the presentdisclosure may be implemented entirely hardware, entirely software(including firmware, resident software, micro-code, etc.) or combiningsoftware and hardware implementation that may all generally be referredto herein as a “module,” “unit,” “component,” “device,” or “system.”Furthermore, aspects of the present disclosure may take the form of acomputer program product embodied in one or more computer readable mediahaving computer readable program code embodied thereon.

A computer readable signal medium may include a propagated data signalwith computer readable program code embodied therein, for example, inbaseband or as part of a carrier wave. Such a propagated signal may takeany of a variety of forms, including electro-magnetic, optical, or thelike, or any suitable combination thereof. A computer readable signalmedium may be any computer readable medium that is not a computerreadable storage medium and that may communicate, propagate, ortransport a program for use by or in connection with an instructionexecution system, apparatus, or device. Program code embodied on acomputer readable signal medium may be transmitted using any appropriatemedium, including wireless, wireline, optical fiber cable, RF, or thelike, or any suitable combination of the foregoing.

Computer program code for carrying out operations for aspects of thepresent disclosure may be written in any combination of one or moreprogramming languages, including an object oriented programming languagesuch as Java, Scala, Smalltalk, Eiffel, JADE, Emerald, C++, C#, VB. NET,Python or the like, conventional procedural programming languages, suchas the “C” programming language, Visual Basic, Fortran 2003, Perl, COBOL2002, PHP, ABAP, dynamic programming languages such as Python, Ruby andGroovy, or other programming languages. The program code may executeentirely on the user's computer, partly on the user's computer, as astand-alone software package, partly on the user's computer and partlyon a remote computer or entirely on the remote computer or server. Inthe latter scenario, the remote computer may be connected to the user'scomputer through any type of network, including a local area network(LAN) or a wide area network (WAN), or the connection may be made to anexternal computer (for example, through the Internet using an InternetService Provider) or in a cloud computing environment or offered as aservice such as a Software as a Service (SaaS).

Furthermore, the recited order of processing elements or sequences, orthe use of numbers, letters, or other designations therefore, is notintended to limit the claimed processes and methods to any order exceptas may be specified in the claims. Although the above disclosurediscusses through various examples what is currently considered to be avariety of useful embodiments of the disclosure, it is to be understoodthat such detail is solely for that purpose, and that the appendedclaims are not limited to the disclosed embodiments, but, on thecontrary, are intended to cover modifications and equivalentarrangements that are within the spirit and scope of the disclosedembodiments. For example, although the implementation of variouscomponents described above may be embodied in a hardware device, it mayalso be implemented as a software only solution, e.g., an installationon an existing server or mobile device.

Similarly, it should be appreciated that in the foregoing description ofembodiments of the present disclosure, various features are sometimesgrouped together in a single embodiment, figure, or description thereoffor the purpose of streamlining the disclosure aiding in theunderstanding of one or more of the various embodiments. This method ofdisclosure, however, is not to be interpreted as reflecting an intentionthat the claimed subject matter requires more features than areexpressly recited in each claim. Rather, claim subject matter lie inless than all features of a single foregoing disclosed embodiment.

We claim:
 1. A system for providing a car hailing service, comprising:at least one storage medium storing a set of instructions; andprocessing circuits in communication with the at least one storagemedium, wherein when executing the set of instructions, the processingcircuits are directed to: classify one or more users into a first usergroup and a second user group using an optimization model, wherein thefirst user group and the second user group correspond to two transportservice adjustments, respectively, the two transport service adjustmentsrelating to the service time, wherein to classify the one or more usersinto the first user group and the second user croup, the processingcircuits are directed to operate the optimization model to: generate arandom number for each of the one or more users: determine whether therandom number for each of the one or more users is greater than athreshold; and in response to the random number for a user being greaterthan the threshold, classify the user into the first user group; obtainbehavior data from terminals of the one or more users in the first usergroup and the second user group, determine, based on the behavior dataof the one or more users, a first value of a parameter regarding thefirst user group and a second value of the parameter regarding thesecond user group using the optimization model, wherein the parameterrepresents at least one of the number of service orders, an orderacceptance rate, or an average response time of the service orders; anddetermine an optimal transport service adjustment based on the firstvalue and the second value.
 2. The system of claim 1, wherein theprocessing circuits are directed to initiate the optimization model toclassify the one or more users into the first user group and the seconduser group when the system obtains service requests from the one or moreusers.
 3. The system of claim 1, the processing circuits are furtherdirected to: in response to the random number for a user being notgreater than the threshold, classify the user into the second usergroup.
 4. The system of claim 1, wherein the parameter includes aparameter in multiple dimensions.
 5. The system of claim 4, wherein theparameter in multiple dimensions relates to order information ofon-demand services.
 6. The system of claim 5, wherein the multipledimensions include a gender, a city, and/or an operation system of theterminal of the one or more users.
 7. The system of claim 1, wherein toobtain behavior data from the terminals of the one or more users in thefirst user group and the second user group, the processing circuits aredirected to: obtain user logs including the behavior data from terminalsof the one or more users using a Hadoop Distribute File System.
 8. Thesystem of claim 1, wherein the processing circuits are further directedto: determine a parameter difference of the parameter regarding thefirst user group and the second user group; determine a reliabilitylevel of the parameter difference; and adjust the optimization modelbased on the parameter difference and the reliability level of theparameter difference.
 9. The system of claim 8, wherein to determine theparameter difference of the parameter regarding the first user group andthe second user group, the processing circuits are directed to:determine a first difference value of the parameter regarding the firstuser group and the second user group.
 10. The system of claim 9, whereinto determine the first difference value of the parameter regarding thefirst user group and the second user group; the processing circuits aredirected to: determine a second difference value; determine a thirddifference value; and determine the first difference value based on thesecond difference value and the third difference value.
 11. The systemof claim 10, wherein to determine the second difference value, theprocessing circuits are directed to: obtain behavior data of the one ormore users associated with one of the two transport service adjustments;determine, based on the behavior data of the one or more usersassociated with the one of the two transport service adjustments, athird value of the parameter regarding the first user group and a fourthvalue of the parameter regarding the second user group using acorrection model; and determine the second difference value based on thethird value and the fourth value.
 12. The system of claim 10, wherein todetermine the third difference value, the processing circuits aredirected to: determine the third difference value based on the firstvalue and the second value.
 13. The system of claim 8, wherein todetermine the reliability level of the parameter difference, theprocessing circuits are directed to: determine, at a preset confidencecoefficient, a confidence interval of the first difference value. 14.The system of claim 8, wherein to determine the reliability level of theparameter difference, the processing circuits are directed to: determinea P value based on the first difference value; compare the P value witha significance value; and determine the reliability level of theparameter difference based on the comparison of the P value with thesignificance value.
 15. A method for providing a car hailing serviceimplemented on a computing device having processing circuits and one ormore storage devices, the method comprising: classifying one or moreusers into a first user group and a second user group using anoptimization model, wherein the first user group and the second usergroup correspond to two transport service adjustments, respectively, thetwo transport service adjustments relating to the service time, whereinclassifying the one or more users into the first user group and thesecond user group comprises: generating a random number for each of theone or more users; determining whether the random number for each of theone or more users is greater than a threshold; in response to the randomnumber for a user being greater than the threshold, classifying the userinto the first user group; and in response to the random number for auser being not greater than the threshold, classifying the user into thesecond user group; obtaining behavior data from terminals of the one ormore users in the first user group and the second user group;determining, based on the behavior data of the one or more users, afirst value of a parameter regarding the first user group and a secondvalue of the parameter regarding the second user group using theoptimization model, wherein the parameter represents at least one of thenumber of service orders, an order acceptance rate, or an averageresponse time of the service orders; and determining an optimaltransport service adjustment based on the first value and the secondvalue.
 16. The method of claim 15, wherein the parameter includes aparameter in multiple dimensions.
 17. The method of claim 15, furthercomprising: determining a parameter difference of the parameterregarding the first user group and the second user group; determining areliability level of the parameter difference; and adjusting theoptimization model based on the parameter difference and the reliabilitylevel of the parameter difference.
 18. A non-transitory computerreadable medium, comprising at least one set of instructions forproviding a car hailing service, wherein when executed by processingcircuits of a computing device, the at least one set of instructionscauses the computing device to perform a method, the method comprising:classifying one or more users into a first user group and a second usergroup using an optimization model, wherein the first user group and thesecond user group correspond to two transport service adjustments,respectively, the two transport service adjustments relating to theservice time, wherein classifying the one or more users into the firstuser group and the second user group comprises: generating a randomnumber for each of the one or more users; determining whether the randomnumber for each of the one or more users is greater than a threshold; inresponse to the random number for a user being greater than thethreshold, classifying the user into the first user group; and inresponse to the random number for a user being not greater than thethreshold, classifying the user into the second user group; obtainingbehavior data from terminals of the one or more users in the first usergroup and the second user group; determining, based on the behavior dataof the one or more users, a first value of a parameter in multipledimension regarding the first user group and a second value of theparameter in multiple dimension regarding the second user group usingthe optimization model, wherein the parameter represents at least one ofthe number of service orders, an order acceptance rate, or an averageresponse time of the service orders; and determining an optimaltransport service adjustment based on the first value and the secondvalue.